Signature Based

Thesis Statement This thesis introduces a new full-text document retrieval model based on the use of document signature access structures combined with ranked retrieval methods. Specifically, it investigates the use of term co-occurence as a measure of similarity between the source document signatures and the signatures of the query terms. The co-occurence of terms is shown to be an attribute characteristic of the signatures which is suitable for ranking documents according to their relevance. Ranking of the documents using term co-occurence in and among the signatures improves the relevance of the retrieval while reducing the size of the index substantially versus inverted file, ranked retrieval methods. This method is based upon the concept that document signatures are contextual windows of meaning. These windows contain passages formed from terms associations which embody the semantics of the underlying text. The comparison of terms co-occurring within and among these windows will be used to rank order the documents decreasingly by relevance. This will produce an indexing method suitable for use on massive document collections. Though document signatures were proven efficient in support of conjunctive queries in Boolean retrieval systems, they are not efficient for the processing of disjunctive queries. Also, Boolean retrieval systems are too complex for effective use by less experienced users. This led to the preeminence of ranked retrieval, which allows the presentation of natural language queries. Ranked retrieval is an enhancement to simple, term based retrieval which only finds documents using terms matching or synonymous with the terms in the query. In the term based retrieval there is no assurance that the most relevant documents are at the top of the 2 list. Ranked retrieval takes the results of the term based retrieval, scores the documents by likely relevance and reorders the documents from most likely relevance to least. Although there are a variety of approaches to performing ranked retrieval, the most widely used method is the context vector representation and the vector space model for comparison scoring. The context vector was introduced by [Gallant91] to manually score document semantics by vectors consisting of an element for each possible term occurrence in the document. The value of the elements in the vector are the weighting factor. Typically the weighting factor used is the frequency of occurrence of the term within the document. A document and a query are compared by comparing their vectors. The most common method for this comparison …